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1.  Introduction 


The  Robotics  Intelligence  Evaluation  Program  (RIEP)  is  a  joint  effort  by  the  U.S.  Army  Aberdeen 
Test  Center  (ATC)  and  U.S.  Army  Research  Laboratory  (ARL)  to  develop  methodologies  to 
evaluate  robotic  behavior  algorithms  that  control  the  actions  of  individual  robots  or  groups  of 
robots  acting  as  a  team  to  perform  a  particular  task.  Although  vehicle  chassis  performance  will 
impact  the  evaluation  of  robotic  behavior  algorithms,  testing  of  intelligent  robotic  platforms 
requires  more  than  the  classic  automotive  tests.  The  tests  and  procedures  required  to  evaluate 
these  algorithms  cross  the  traditional  boundaries  between  operational  and  developmental  testing 
and  between  hardware  and  software  testing.  Developmental  testing  evaluates  behavior  algorithms 
using  performance  specifications  contained  in  the  relevant  system  contract.  However,  these 
specifications  define  parameters  such  as  chassis  performance,  sensor  detection  levels,  and  low 
level  behaviors  which  might  include  maximum  on-road  speed,  minimum  obstacle  height  detection, 
maximum  range  at  which  system  can  detect  a  human-sized  obstacle,  and  so  on.  Specifications  for 
higher  level  behaviors  will  likely  be  less  well  defined.  In  some  sense,  evaluating  behavior 
algorithms  is  similar  to  evaluating  new  tactics,  techniques,  and  procedures  (TTPs)  for  manned 
systems.  Battlefield  simulation  tools  are  generally  used  to  allow  researchers  to  look  at  the  effect  of 
TTPs  on  combat  performance  measures,  and  these  same  tools  are  key  to  the  evaluation  of 
intelligent  robotic  systems.  Behavior  algorithm  performance  is  affected  by  software  design  as  well 
as  vehicle  design,  but  these  algorithms  are  generally  driven  by  on-board  sensors  and  external 
information  sources  such  as  human  operators  or  situational  information  systems  that  supply  the 
input  that  determines  specific  actions  within  the  behavior  algorithm. 

There  are  four  aspects  of  the  RIEP  process.  First  is  a  thorough  analysis  of  the  task  addressed  by 
the  behavior  algorithm.  Once  the  task  is  understood,  the  second  aspect  of  the  methodology  is  to 
find  a  simulation  environment  that  is  “rich”  enough  to  simulate  the  task  and  test  the  proposed 
algorithm.  The  third  aspect  of  RIEP  is  to  detennine  a  suitable  method  to  link  the  proposed 
algorithm  to  the  simulation  environment.  The  last  aspect  of  the  program  is  to  design  a  set  of 
meaningful  tests  and  perfonnance  measures  to  evaluate  a  proposed  behavior  algorithm. 

The  first  step  of  the  REIP  process  is  a  thorough  analysis  of  the  task  addressed  by  the  behavior 
algorithm.  The  emphasis  is  on  the  task,  not  the  algorithm,  since  many  algorithms  can  be 
proposed  to  solve  a  specific  task.  Behavior  algorithms  are  being  designed  to  perform  tasks 
ranging  from  system  health  monitoring  to  battlefield  missions  such  as  area  reconnaissance.  The 
four-dimensional  real-time  control  system  (4D-RCS),  an  architecture  for  designing  robot  control 
systems,  provides  a  convenient  way  to  rank  task  complexity  and  to  look  at  the  resolution  of 
information  required  by  each  task.  Figure  1  provides  a  diagram  of  the  4D-RCS  architecture. 

Each  level  of  the  architecture  contains  three  components:  an  executor  that  accepts  plans  and 
information  from  the  next  higher  level;  a  planner  that  directs  the  actions  of  the  robot  and  sends 
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tasks  to  the  next  lower  level;  and  a  world  model  that  contains  sensor  information.  At  the  servo 
level,  the  robot  controls  the  actuators  that  move  the  vehicle  and  subsystems  such  as  cameras  on 
board  the  robot  itself.  The  robot  acts  quickly,  typically  on  a  microsecond  scale,  on  information 
provided  by  on-board  sensors.  The  primitive  level  involves  low-level  vehicle  tasks  such  as 
obstacle  avoidance,  image  processing,  etc.  The  subsystem  level  combines  tasks  into  important 
activities,  such  as  autonomous  mobility  or  reconnaissance.  At  the  vehicle  level,  the  robot 
performs  functions  that  are  tactically  useful  on  a  battlefield.  At  the  section  level,  these  functions 
extend  to  include  tasks  that  are  performed  by  groups  of  robots.  The  4D-RCS  continues  beyond 
the  levels  shown  in  figure  1;  the  higher  levels  resemble  levels  of  military  organizations. 

Tasks  can  involve  multiple  levels  of  the  4D-RCS  structure.  For  instance,  a  reconnaissance 
mission  is  a  section-level  task  that  involves  a  team  of  robots  that  provides  information  about  a 
designated  area  of  the  battlefield.  Each  robot  performs  independent  vehicle-level  behaviors  that 
contribute  to  the  overall  mission.  These  vehicle-level  tasks  can  be  subdivided  into  three  distinct 
subsystems:  mobility,  sensing,  and  weapon  control.  Autonomous  mobility  involves  map-based 
planners  and  obstacle-avoidance  systems.  Autonomous  sensing  may  involve  several  sensors  and 
fusion  algorithms  that  combine  information  from  multiple  sensor  sources.  Once  we  understand 
the  task,  we  can  identify  critical  elements  of  the  environment  needed  to  support  an  evaluation  of 
algorithm  designed  to  address  the  task.  Generally,  we  need  to  consider  terrain  topography, 
composition,  surface  features,  weather  elements  such  as  wind  and  rain,  and  the  simulation  tool. 
The  world-model  blocks  from  figure  1  provide  a  rough  estimate  of  the  level  of  detail  required  to 
stimulate  tasks  at  each  level  of  the  architecture.  Section-level  behavior  tasks  depend  on  very 
crude  infonnation  such  as  the  location  of  lakes  or  canopy  areas.  At  the  subsystem  level,  tasks 
depend  on  detailed  environmental  information  such  as  the  location  of  ditches  and  individual 
trees.  The  primitive  level  requires  even  more  detail,  such  as  the  location,  shape,  and  composition 
of  terrain  features. 

The  choice  of  the  simulation  tool  can  also  be  tied  to  the  4D-RCS  structure.  Effective  testing  of 
section-level  behaviors  requires  a  simulation  tool  capable  of  representing  the  position  of 
individual  vehicles  and  modeling  engagements  between  two  or  more  forces.  Also,  it  may  be 
desirable  to  model  at  least  some  of  the  command  and  control  (C2)  structure  for  the  unit  that 
executes  the  behavior  algorithm.  Sensor  information  is  represented  as  polygonal  and  linear 
“map”  features  that  become  known  as  the  vehicle  moves  around  in  the  simulated  world. 
Subsystems  such  as  the  autonomous  mobility  system  require  simulation  tools  that  can  model  the 
position  and  orientation  of  each  vehicle.  Sensor  models  provide  infonnation  about  the 
immediate  vicinity  of  the  vehicle.  Visual  sensor  performance  models  interact  with  objects 
approximately  10  to  20  inches  in  radius.  The  primitive  level  requires  tools  that  can  model  the 
position  and  orientation  of  vehicle  components.  At  this  level,  it  becomes  important  to  model  the 
process  of  sensing  the  environment. 
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Figure  1.  The  4D-RCS  robotics  control  architecture. 


Linking  the  behavior  algorithm  that  is  undergoing  test  to  the  virtual  world  can  be  accomplished  in 
two  ways.  The  first,  a  dissection  approach,  is  to  rewrite  the  behavior  algorithm  as  an  element  of 
simulation  tool  used  in  the  analysis.  If  this  approach  is  carefully  executed,  it  leads  to  detailed 
understanding  of  the  algorithm  undergoing  test,  the  sensory  information  required  to  drive  the 
algorithm,  and  the  relationship  between  the  behavior  algorithm  and  the  host  robotic  platform. 

This  approach  is  useful  in  building  a  behavior  algorithm  for  complex  force-on-force  models.  It  is 
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also  useful  for  modularized  behavior  algorithms  that  can  be  separated  into  component  parts  with 
the  4D-RCS  architecture  as  a  guide.  As  a  module  in  the  behavior  is  changed,  that  module  of  the 
simulation  can  be  revised.  Researchers  can  use  these  systems  to  study  the  response  of  high-level 
behaviors  to  different  levels  of  performance  of  the  sub-behaviors.  To  be  successful,  this  approach 
requires  close  collaboration  between  the  behavior  developer  and  the  behavior  evaluator. 


The  second  method,  a  “black  box”  approach,  is  to  link  the  behavior  system  embedded  in  the  robot 
with  the  virtual  environment.  Often,  this  is  the  preferable  approach  since  the  algorithm  cannot  be 
modified  during  the  process.  Figure  2  shows  a  graphic  that  illustrates  this  approach.  The  behavior 
algorithm  in  the  middle  block  is  the  system  undergoing  test.  Information  from  the  virtual  world 
provides  the  behavior  algorithm  with  simulated  sensor  input.  Sensor  models  in  the  virtual 
environment  must  be  capable  of  supplying  sensor  information  at  the  level  of  detail  required  by  the 
behavior  algorithm.  Commanding  entities  provide  C2  information  by  providing  orders,  routes,  and 
situational  awareness  overlays.  Information  also  flows  from  the  behavior  algorithm  to  other 
entities  in  the  exercise.  Both  of  these  connections  depend  on  a  communica-tion  model  that  models 
the  flow  of  information  in  and  out  of  the  behavior  algorithms.  The  key  to  success  for  this  approach 
is  to  define  the  interfaces  between  behavior  algorithm  and  the  virtual  environment. 


commands  from  real 
and  simulated  entities 


terrain  elevation  and 
mapped  terrain  features 


Figure  2.  Embedding  a  behavior  algorithm  in  a  virtual  environment. 
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The  last  step  of  the  RIEP  is  to  design  a  set  of  meaningful  tests  to  characterize  behavior  algorithm 
performance  in  battlefield  environments.  Test  scenarios  need  to  include  environmental 
challenges  that  allow  researchers  to  assess  the  robustness  and  reliability  of  the  behavior 
algorithm  undergoing  test.  Testing  low-level  behavior  algorithms  such  as  autonomous  mobility, 
resembles  traditional  testing,  i.e.,  the  virtual  environment  is  populated  with  specific  features 
designed  to  test  the  response  of  the  system.  Often,  these  tests  can  be  tied  directly  to  a  contract 
specification  or  to  an  operational  requirements  document  (ORD).  Higher  level  behaviors  need  to 
be  tested  in  the  context  of  force-on-force  scenarios.  ORDs  and  contract  specifications  are  often 
vague  about  perfonnance  expectations  for  high-level  behaviors,  so  it  is  useful  to  examine  a  series 
of  force-on-force  scenarios  designed  to  find  perfonnance  limitations.  Early  in  the  development 
cycle,  well-designed  force-on-force  scenarios  can  help  focus  the  development  effort  on 
frequently  occurring  issues  and  away  from  rarely  occurring  issues.  One  issue  that  needs  to  be 
addressed  with  high-level  behaviors  is  the  behavior  undergoing  test.  One  approach  is  to  regard  a 
test  of  a  high-level  behavior  as  a  test  of  all  the  underlying  algorithms.  Since  most  of  the 
subsystems  are  not  yet  finalized,  we  may  need  methods  to  isolate  the  performance  of  high-level 
algorithms  from  the  lower  level  algorithms. 

The  RIEP  process  is  being  refined  through  a  series  of  case  studies,  beginning  with  subsystem- 
level  behaviors  and  ending  with  the  evaluation  of  a  mixed  human-robot  reconnaissance  behavior. 
In  this  first  year  of  the  process,  we  have  chosen  to  examine  vehicle  rollover  prevention 
algorithms.  Rollover  prevention  is  critical  for  the  safety  of  manned  and  unmanned  systems.  It  is 
also  specified  by  the  ORD  for  some  near-term  robotic  systems. 

The  remainder  of  this  document  is  a  discussion  of  the  progress  of  RIEP  during  the  first  year.  We 
begin  with  a  discussion  of  vehicle  rollover  and  prevention  systems.  Most  of  the  information  for 
these  sections  comes  from  the  automotive  industry  where  anti-rollover  technologies  are  becoming 
important  safety  features.  Sections  4  and  5  discuss  the  current  measures  and  tests  used  by 
automotive  safety  engineers  and  some  of  the  additional  elements  that  need  to  be  considered  for 
robotic  systems.  We  discuss  several  simulation  tools  applicable  to  vehicle  rollover  studies. 
Finally,  we  make  recommendations  for  virtual  and  physical  tests  of  anti-rollover  systems. 


2.  Vehicle  Rollover 


Although  we  consider  primarily  “car-like”  robots  on  relatively  flat  surfaces  in  this  research,  it  is 
useful  to  provide  a  general  definition  of  vehicle  rollover  that  applies  to  traditional  wheeled  and 
tracked  vehicles  as  well  as  vehicles  with  legs,  large  mobile  manipulators,  outriggers,  or  other 
novel  characteristics  operating  in  a  wide  variety  of  terrains.  Consider  an  arbitrary  ground  vehicle 
that  has  n  points  of  contact  with  the  ground.  These  n  contact  points  form  a  support  polygon  for 
the  ground  vehicle.  A  rollover  or  “tip-up”  occurs  when  the  vehicle’s  center  of  gravity  (c.g.) 
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moves  far  enough  outside  the  support  polygon  to  cause  the  vehicle  to  rotate  around  one  of  the 
polygon  sides  (Papadopolos  &  Rey,  1996).  In  automotive  applications,  cars  typically  roll 
laterally  around  either  the  left  or  right  set  of  wheels. 

Dynamic  rollovers  can  be  divided  into  two  classes:  tripped  and  untripped.  Tripped  rollovers  are 
initiated  by  a  mechanism  such  as  a  curb,  guardrail,  soft  soil  or  road  edge.  Tripped  rollovers  are 
often  preceded  by  a  loss  of  vehicle  control  that  allows  the  vehicle  to  leave  the  road  surface. 
Untripped  rollovers  are  initiated  by  vehicle  maneuvers  on  a  nonnal  road  surface.  According  to 
the  National  Automotive  Sampling  System  Crashworthiness  Data  System  (NASS  CDS) 
maintained  by  the  National  Highway  Traffic  Safety  Administration  (NHTSA),  less  than  5%  of 
the  rollover  crashes  are  untripped  (Hertz,  1999).  Despite  their  low  frequency  of  occurrence, 
much  of  NHTSA’s  research  efforts  focuses  on  developing  extreme  steering  maneuvers  to  test 
untripped  rollovers  (Forkenbrock,  Garrott,  Heitz,  &  O’Harra,  2002;  Forkenbrock,  O’Harra,  & 
Elasser,  2004;  Howe,  Garrott,  Forkenbrock,  Heydinger,  &  Lloyd,  2001).  Untripped  rollovers  are 
directly  related  to  controllable  factors  such  as  vehicle  design  and  driver  behavior.  Tripped 
rollovers  are  also  related  to  vehicle  design  and  driver  behavior,  but  any  vehicle  will  roll  over  if  it 
impacts  a  suitable  tripping  mechanism  with  sufficient  lateral  velocity. 


3.  Rollover  Prevention 


Unmanned  systems  are  still  in  the  development  stage  of  their  life  cycle.  Consequently,  we  do  not 
have  a  specific  anti-rollover  system  to  test.  However,  anti-rollover  systems  developed  for  ground 
robots  are  likely  to  be  similar  to  those  developed  for  the  automotive  and  commercial  vehicle 
industries.  These  industries  are  very  interested  in  developing  methods  to  predict  and  prevent 
vehicle  rollover.  Their  development  efforts  include  “driver-based”  strategies  designed  to  reduce 
the  probability  of  encountering  a  near-rollover  condition  and  vehicle-based  systems  such  as 
rollover  warning  sensors,  suspension  system  components  such  as  anti-roll/sway  bars  and  vehicle 
stability  control  (VSC)  systems  designed  to  recover  from  a  near-rollover  condition.  Each  of  these 
efforts  is  potentially  applicable  to  unmanned  systems. 

Rollovers  can  be  prevented  by  training  drivers  to  maintain  vehicle  control.  Tire  maintenance 
affects  the  controllability  of  the  vehicle.  Cargo  loading  strategies  can  change  the  c.g.  affecting 
rollover  propensity.  Drivers  can  be  taught  emergency  maneuvers  that  help  them  maintain  control 
of  the  vehicle.  Some  driver  education  translates  to  the  control  of  unmanned  vehicles.  Vehicle 
maintenance  and  loading  is  similar  for  manned  and  unmanned  systems.  For  tele-operated  systems, 
the  operator  can  be  trained  to  avoid  excessive  speed  and  steering.  For  autonomous  systems,  the 
robot  planning  algorithms  can  consider  vehicle  stability  as  a  factor  as  it  plans  its  path. 

In  the  last  decade,  many  groups  have  begun  to  develop  rollover  warning  systems  for  passenger 
and  commercial  vehicles.  Under  the  Intelligent  Highway  Program,  “smart”  signs  can  warn 
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trucks  of  impending  rollover  by  measuring  the  speed  of  the  truck.  Chen  and  Peng  (1999)  have 
developed  a  time-to-rollover  warning  algorithm  that  can  warn  drivers  of  impending  rollovers, 
based  on  the  vehicle  yaw  rate.  Rey  and  Papadopoulos  (1997)  have  developed  a  warning 
algorithm  applicable  to  forestry  equipment  with  booms,  cranes,  or  other  manipulators. 
Automobile  manufacturers  have  also  developed  rollover  sensors.  However,  most  of  these 
sensors  are  used  to  detennine  when  protective  devices  such  as  air  bags  should  be  deployed  to 
protect  the  vehicle  occupants.  Warning  systems  can  allow  operators  of  tele-operated  systems  to 
correct  unsafe  vehicle  operations.  They  could  also  provide  an  audio  or  visual  rollover  warning  to 
troops  in  the  vicinity  of  an  unmanned  system.  The  sensors  and  algorithms  of  the  rollover 
warning  systems  can  be  incorporated  into  autonomous  planning  systems. 

Suspension  system  components  such  as  anti-roll  or  sway  bars  control  the  body  roll  in  cornering 
maneuvers.  Another  vehicle  design  technology  (active  rear  steering)  is  designed  to  increase 
vehicle  maneuverability  at  high  speeds.  Hac  (2002)  examined  the  effect  of  these  vehicle  design 
choices  on  rollover  propensity. 

VSC  systems  are  designed  to  assist  drivers  in  maintaining  steering  control  of  their  vehicles. 

These  systems  use  yaw  rate  sensors  to  compare  the  actual  path  of  a  vehicle  to  its  intended  path. 
Automatic  braking  can  be  applied  independently  to  each  wheel  to  compensate  for  excessive 
oversteer  or  understeer.  VSC  indirectly  prevents  rollover  by  keeping  the  vehicle  on  the  road 
surface.  There  are  some  new  (2005)  systems  that  attempt  to  directly  prevent  untripped  rollover 
by  incorporating  roll  rate  and  lateral  acceleration  sensors.  Again,  the  control  mechanism  is  to 
decelerate  the  vehicle  and  to  reduce  the  sharpness  of  the  turn.  Unmanned  systems  can  benefit 
from  VSC  technology.  Tele-operated  systems  could  use  the  technology  to  compensate  for 
system  latency,  lack  of  driving  awareness,  or  over-eager  operators.  Autonomous  systems  could 
use  VSC  technology  to  monitor  and  adjust  the  vehicle  yaw  rate;  this  may  be  particularly 
important  on  loose  soil  or  ice-covered  surfaces. 


4.  Current  Measures  and  Tests 


Highway  rollover  crashes  are  a  major  safety  concern.  Consequently,  automotive  safety  engineers 
in  NHTSA,  the  automotive  industry,  insurance  industry,  and  consumers’  groups  have  developed 
tests  designed  to  measure  the  rollover  propensity  of  passenger  and  commercial  vehicles.  Their 
concerns  as  they  develop  these  tests  are  similar  to  ours,  namely,  test  repeatability,  relevance  to  real 
driving  issues,  and  crash  rate  predictability.  Some  of  the  tests  are  static  measures  based  on  vehicle 
parameters  such  as  vehicle  weight  and  the  location  of  the  c.g.  Newer  tests  are  dynamic  maneuvers 
designed  to  look  at  vehicle  perfonnance  in  a  series  of  maneuvers  designed  to  mimic  worst  case 
emergency  maneuvers. 
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A  common  measure  of  rollover  propensity  is  the  static  stability  factor  (SSF).  This  measure  is 
derived  from  the  physics  of  rigid  bodies.  Consider  a  vehicle  to  be  a  rigid  body  with  mass  m  and 
width  t  moving  along  an  arc  of  radius  r  at  a  speed  v  on  a  flat  surface.  Suppose  that  the  height  of 
the  c.g.  for  this  vehicle  is  h.  The  vehicle  rolls  on  its  side  if  the  lateral  forces  given  in  the  right- 
hand  side  of  equation  1  exceed  the  vertical  forces  given  on  the  left-hand  side. 


mg 


(1) 


Rearranging  equation  1  provides  the  definition  for  the  SSF: 


t  _  v2 
2  h  rg 


(2) 


The  left-hand  side  of  equation  2,  which  is  defined  to  be  the  SSF,  depends  only  on  measurable 
vehicle  parameters:  the  c.g.  and  vehicle  width.  The  right-hand  side  of  the  equation  gives  a 
practical  safety  formula  specifying  safe  combinations  of  speed  and  turn  radius.  Points  along  the 
spiral  path  shown  in  figure  3  have  been  labeled  with  the  rollover  speeds  for  two  different  values 
of  the  SSF.  The  circled  numbers  indicate  the  turn  radius  of  the  spiral  path  in  meters.  The  blue 
numbers  indicate  rollover  speeds  for  vehicles  with  SSF  =  1.0.  The  red  numbers  indicate 
rollovers  for  a  vehicle  with  SSF  =  1.5.  Figure  4  shows  a  size  comparison  between  vehicles  with 
SSF  =  1.0  and  SSF  =  1.5. 


Equation  2  applies  to  vehicle  on  flat  ground.  A  more  general  equation  relates  the  lateral 
acceleration  to  the  SSF  and  the  side  slope  angle,  (j>. 

SSF  -  tan  <p  v 2 
SSF  tan  <p  +  1  rg 


Other  static  rollover  propensity  measures  are  the  tilt  table  ratio  (TTR),  the  side  pull  ratio  (SPR), 
and  the  critical  sliding  velocity  (CSV).  In  the  tilt  table  test,  the  vehicle  is  placed  on  a  table  and 
tilted  laterally  until  the  front  and  rear  wheels  on  the  uphill  side  lift.  The  tangent  of  the 
longitudinal  table  angle  is  the  TTR.  The  SSR  is  the  ratio  of  the  vehicle  weight  to  the  lateral  force 
required  to  cause  two-wheel  lift.  The  CSV  is  lateral  velocity  required  to  cause  two-wheel  lift 
when  the  vehicle  strikes  a  tripping  obstacle  such  as  a  curb.  It  can  be  computed  with  the  use  of 
the  vehicle  mass,  M,  the  vehicle  width,  T,  the  height  of  the  c.g.,  H,  and  the  roll  moment  of 
inertial  about  the  c.g.,  Ixx  (Hey dinger  et  ah,  1999). 


with 
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Huang  (Huang,  2002)  has  a  detailed  derivation  of  the  CSV  equation. 


Rollover  Speeds  (mps) 


-10  -SO  5  10 


X  Meters 


Figure  3.  A  spiral  path  showing  the  required  rollover  velocity. 


Rollover  Ratings 

Static  Stability  Factor 


1.50  1.00 


SSF  =  -C 
2h 


Figure  4.  Hypothetical  vehicles  with  SSF  =  1.5  and  SSF  =  1.0  (NHTSA,  2005). 

Since  the  static  measures  can  be  collected  in  laboratory  settings,  these  tests  are  considered  very 
repeatable.  However,  these  measures,  especially  the  SSF,  have  been  criticized  for  being  too 
simplistic.  The  SSF  and  the  CSV  treat  the  vehicle  as  a  solid  block,  giving  no  credit  to  the 
stabilizing  influence  of  the  suspension  system.  There  is  also  some  concern  that  manufacturers 
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can  optimize  their  vehicles  to  “score”  well  on  the  CSV  without  any  improvement  in  the  safety  of 
the  vehicle  (NHTSA,  2004).  Since  they  are  static  measures,  they  cannot  be  used  to  evaluate  the 
performance  of  VSC  systems.  However,  the  SSF  correlates  well  with  existing  accident  statistics. 
Also,  according  to  NHTSA,  the  static  measures  are  the  best  predictors  of  vehicle  performance 
during  a  potential  trip. 

Dynamic  rollover  propensity  maneuvers  allow  the  vehicle  performance  to  be  tested  in  situ. 

They  allow  the  complex  interactions  between  the  suspension  system  and  the  vehicle  chassis  and 
environmental  factors  such  as  road  surface  and  cross  winds.  Test  maneuvers  can  also  evaluate 
the  effect  of  VSC  systems  on  vehicle  rollover.  However,  the  very  elements  that  make  test 
maneuvers  relevant  to  real-world  driving  make  repeatability  difficult.  Right  now,  there  is  not 
enough  data  to  determine  the  relationship  between  driving  maneuver  performance  and  rollover 
accident  statistics. 

The  driving  maneuvers  used  by  automotive  safety  engineers  include  slowly  increasing  steer, 
j-turn,  and  fishhook  (Howe,  Garrott,  &  Forkenbrock,  2001).  In  a  slowly  increasing  steer 
maneuver,  the  driver  keeps  the  speed  constant  and  gradually  increases  the  steering  wheel  angle. 
In  a  related  maneuver,  the  steering  wheel  angle  is  held  constant  while  the  speed  is  gradually 
increased.  The  primary  use  of  these  maneuvers  is  to  characterize  the  responsiveness  of  the 
vehicle  and  to  determine  the  maximum  obtainable  lateral  accelerations.  The  j-turn  is  a  sudden 
large  turn  similar  to  the  maneuvers  required  to  negotiate  a  cloverleaf  ramp.  The  fishhook 
maneuver  involves  a  large  steering  wheel  angle  change  followed  by  a  large  steering  wheel  angle 
change  in  the  opposite  direction.  It  is  analogous  to  maneuvers  to  avoid  obstacles  in  the  roadway. 

The  use  of  steering  and  braking  machines  increases  the  repeatability  of  the  maneuver  tests. 
Performing  these  maneuvers  on  a  test  track  is  still  a  risk  to  the  drivers  and  the  vehicles. 

Roadway  simulators  such  as  the  one  at  ATC  can  reduce  risk  to  the  test  article  and  human 
participants  and  increase  test  repeatability  (Connon,  2005). 

Crashworthiness  studies  use  tests  designed  to  deliberately  induce  a  rollover.  These  include 
driving  onto  corkscrew  ramps,  sliding  into  tripping  hazard  such  as  posts  or  gravel  pits,  or  driving 
into  a  hole.  A  common  use  for  these  tests  has  been  to  examine  occupant  safety.  Recently,  Viano 
and  Parenteau  (2004)  have  developed  a  series  of  tests  to  define  rollover  sensor  requirements  for 
applications  such  as  air  bag  deployment.  According  to  their  research,  the  results  of  these 
instrumented  repeatable  tests  mimic  about  90%  of  real-world  rollover  situations. 


5.  Roll-Over  Testing  for  Robotic  Systems 


The  previous  section  described  tests  applicable  to  passenger  vehicles  driven  on  highway  systems. 
These  tests  are  applicable  to  robotic  systems,  but  there  are  additional  factors  that  need  to  be 
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considered  for  robots.  Situational  awareness,  communication  delays,  perception,  and  path 
planning  all  have  a  potential  impact  on  rollover  propensity. 

The  robots  considered  in  this  research  are  either  tele-operated  or  autonomously  driven.  Each  of 
these  control  mechanisms  has  its  own  challenges  with  respect  to  evaluating  rollover  propensity. 
Tele-operated  systems  depend  on  the  human  operator  to  perceive  and  avoid  navigational  hazards. 
These  systems  require  a  very-high-bandwidth,  low-latency,  and  highly  reliable  communications 
system.  Delay  in  the  communications  system,  coupled  with  the  limited  perception  available  to 
the  operator,  can  lead  to  vehicle  rollover.  A  study  of  tele-operated  system  accidents  from  early 
1990s  found  that  60%  of  the  accidents  were  rollovers  (McGovern,  1991).  Generally,  the 
operators  had  only  limited  awareness  of  the  vehicle  orientation  and  were  unable  to  sense  near¬ 
rollover  conditions  before  they  became  irreversible.  More  recent  work  in  the  field  of  urban 
search  and  rescue  (USAR)  robots  found  that  situational  awareness  remains  an  important  issue 
(Burke,  Murphy,  Coovert,  &  Riddle,  2004). 

Autonomous  systems  use  planners  to  process  a  priori  infonnation  such  as  digital  maps  and 
acquired  information  such  as  sensor  readings  to  navigate.  In  systems  using  hierarchical  control 
architecture,  the  planner  is  divided  into  subsystems  that  operate  at  different  frequencies.  The 
mission  planner  translates  the  overall  mission  objectives  into  tasks  and  specific  destinations  for 
the  robot;  the  mission  plan  is  revised  infrequently  as  the  mission  changes.  The  navigator  does 
global  path  planning  based  on  an  a  priori  map  and  situational  awareness  information;  paths  are 
replanned  as  tactical  information  (such  as  known  enemy  positions)  changes.  The  pilot  does 
moment-to-moment  trajectory  planning.  It  monitors  current  position  and  pose  (roll,  pitch,  yaw) 
using  a  variety  of  means  such  as  an  IMU  (inertial  measurement  unit),  GPS  (global  positioning 
system),  and  odometry  with  estimates  from  all,  combined  by  a  Kalman  filter  or  something 
similar.  Trajectories  change  frequently  as  the  perception  system  acquires  new  information  about 
the  environment  (Board  on  Army  Science  and  Technology  [BAST],  2002). 

Figure  5  shows  a  contour  map  and  three  of  many  possible  paths  between  points  A  on  the  upper 
left  edge  of  the  map  and  point  B  in  the  lower  right  region  of  the  map.  A  robotic  system  chooses 
the  “best”  path  by  minimizing  the  cost  of  each  path.  Costs  include  travel  time,  exposure  to 
known  enemy  positions,  vehicle  safety,  and  other  factors.  In  this  example,  the  red  path  may 
minimize  exposure  to  an  enemy  in  the  upper  right  area  of  the  map,  but  it  may  not  be  safe  for  the 
robot  to  travel  along  the  steep  contours  of  the  hills.  Plans  evolve,  as  the  blue  and  green  plans 
illustrate.  In  this  illustration,  the  blue  path  incorporated  additional  sensor  information  that 
became  available  when  the  robot  reached  the  middle  of  the  map. 

Path  planning  decisions  can  impact  rollover  propensity.  Planners  that  do  not  incorporate  system 
safety  into  the  cost  function  may  be  more  likely  to  roll  than  planners  that  consider  vehicle 
orientation  as  a  cost  variable.  Exercising  the  planners  on  a  variety  of  realistic  battlefields  allows 
researchers  to  evaluate  the  rollover  sensitivity  of  the  planning  process. 
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Figure  5.  Paths  between  point  A  at  the  upper  left  edge  of  the  map  and  point  B  in  the 
lower  right  region  of  the  map. 


6.  Simulation  Tools 


Modeling  and  simulation  programs  are  becoming  common  tools  in  automotive  safety  engineering 
In  this  section,  we  discuss  the  use  of  simulation  tools  to  supplement  the  laboratory  measurements 
and  field  testing  for  rollover  propensity.  Three  common  simulation  tools,  PC-Crash1,  TruckSim, 
and  CarSim2,  are  used  in  the  rollover  literature.  We  also  discuss  two  tools  developed  specifically 
for  military  use:  Vehicle  Dynamics  Mobility  Server  (VDMS)  and  One  Semi-Automated  Forces 
(OneSAF).  VDMS  is  similar  to  the  commercial  automotive  simulation  packages  used  in  the 
literature.  OneSAF  is  a  battlefield  simulation  tool. 

PC-Crash  is  a  momentum-based  simulation  tool  which  is  used  in  accident  reconstruction.  It  can 
simulate  scenarios  involving  as  many  as  32  entities  including  cars,  trucks,  pedestrians,  and 
bicyclists.  Vehicles  are  described  by  geometric  properties  such  as  size,  weight  and  the  c.g., 
suspension  properties  such  as  stiffness  and  tire  characteristics,  power  train  properties,  and 


1  PC-Crash  is  a  trademark  of  Dr.  Steffan  Datentechnik  Ges.m.b.H. 

-TruckSim  and  CarSim  are  trademarks  of  Mechanical  Simulations  Corporation. 
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moments  of  inertia.  Aspects  of  the  simulated  environment  such  as  wind  speed,  obstacle 
locations,  and  road  surface  characteristics  can  be  modified.  In  addition,  PC-Crash  has  two 
different  driver  models  to  control  the  vehicles.  Often,  PC-Crash  is  combined  with  a  finite 
element  model  (e.g.,  LS-DYNA3,  MaDymo4,  and  PAM-Crash5)  to  look  at  vehicle  and  occupant 
damage. 

Viano  and  Parenteau  (2004)  used  PC-Crash  to  help  develop  test  procedures  and  equipment  to 
support  their  work  on  rollover  sensors.  This  tool  was  used  to  simulate  various  tripped  rollover 
maneuvers  and  to  set  experimental  parameters  such  as  impact  speed  and  driving  conditions  for 
the  field  rollover  tests.  Since  this  tool  is  often  used  in  court  cases,  there  have  been  several 
validation  studies.  Cliff  and  Montgomery  (1996)  conducted  some  early  validation  studies. 

Later,  Cliff  and  Moser  (2001)  compared  PC-Crash  results  to  staged  vehicle  crashes.  Recently, 
Gopal,  Baron,  and  Shah  (2004)  published  some  initial  validation  studies  comparing  PC-Crash 
results  to  controlled  laboratory  rollover  tests. 

CarSim  and  TruckSim  have  been  used  for  more  than  20  years  to  study  the  dynamics  of 
automobiles  and  trucks.  CarSim  simulates  the  dynamic  behavior  of  race  cars,  passenger  cars, 
light  trucks,  and  utility  vehicles.  Typically,  a  vehicle  is  described  as  a  14-degree-of-freedom 
(DOF)  system.  The  chassis  has  three  rotational  and  three  translational  degrees  of  freedom,  and 
each  wheel  can  rotate  on  its  axle  and  move  vertically.  TruckSim  is  a  similar  product  designed 
for  analyzing  commercial  trucks  and  articulated  vehicles.  New  vehicle  models  are  created  with 
the  CarSim-TruckSim  user  interfaces  provided  by  the  simulation  package  or  by  models  created 
with  the  MATLAB6  and  SIMULINK  packages  commonly  used  as  design  tools  in  the  automotive 
industry.  The  road  surface  can  be  modified  to  represent  many  different  environments,  including 
off-road  areas. 

Since  CarSim  is  a  multi-component  model,  it  has  been  used  to  study  the  interactions  of  the 
suspension  systems  with  the  chassis  (Sharp  &  Bettella,  2001).  By  linking  the  vehicle  model  with 
external  simulation  tools,  other  researchers  have  used  the  simulation  to  develop  vehicle  stability 
control  algorithms  and  sensors  (Eisele  &  Peng,  2000).  Anwar  (2004)  used  the  model  to  develop 
a  traction  control  algorithm.  Chen  and  Peng  have  used  it  to  investigate  rollover  warning  systems 
for  trucks.  It  has  been  linked  to  motion  simulators  and  it  can  support  human  driver  studies. 

Ungoren,  Peng,  and  Tseng  (2004)  have  also  used  CarSim  to  iteratively  develop  a  series  of  worst 
case  maneuvers  to  test  vehicle  stability  control  systems  for  sport  utility  vehicles. 

There  have  been  some  validation  studies  as  well.  Alonso  provides  a  comparison  between  a 
general  14-dof  vehicle  model  and  field  maneuvers  (Alonso,  2005).  Ungoren  et  al.  (2004) 
compared  the  performance  of  a  TruckSim  Jeep  Cherokee  model  to  field  data  for  banked  turn 

LS-DYNA  is  a  trademark  of  Livermore  Software  Technology  Corp. 

4MaDymo  is  a  trademark  of  the  Netherlands  Organization  for  Scientific  Research. 

5PAM-Crash  is  a  trademark  of  the  ESI  Group. 

6MATLAB  and  SIMULINK  are  registered  trademarks  of  the  MathWorks. 
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maneuvers.  Kalwauchi  et  al.  (2005),  found  a  favorable  comparison  between  CarSim  models  and 
actual  car  performance  on  an  oval  test  track. 

Another  simulation  tool  to  consider  is  Tank- Auto  motive  and  Armaments  Command  (TACOM)- 
TACOM  Research,  Development,  and  Engineering  Center  (TARDEC’s)  VDMS  (Brudnak, 

Nunez,  &  Reid,  2002).  VDMS  represents  a  vehicle  as  a  6-DOF  chassis  and  four  vehicle  comers 
that  represent  the  wheels  and  suspension  system  components  at  each  wheel.  New  vehicle  models 
are  created  by  commercial  packages  such  as  SimCreator  or  MATLAB  or  SIMULINK.  VDMS 
represents  several  aspects  of  vehicle  dynamics  including  power  and  traction  limits  on  mobility 
because  of  terrain  slope  and  composition,  vibration  effects  because  of  surface  roughness,  and 
vehicle  instabilities  such  as  rollover.  Vehicle  control  is  provided  by  an  external  interface  such  as 
joystick  for  tele-operated  systems  or  by  external  control  algorithms  for  autonomous  systems. 

The  model  operates  at  200  to  500  Hz  which  could  support  interactions  with  vehicle  attitude 
sensors  and  anti-rollover  algorithms  that  must  operate  quickly  to  stabilize  the  vehicle.  VDMS 
can  be  used  as  a  stand-alone  model  or  as  part  of  a  distributed  modeling  environment.  VDMS  is  a 
component  of  the  Modeling  Architecture  for  Technology  and  Research  Experience  (MATREX) 
federation  of  models  developed  by  the  U.S.  Army  Research  Development  and  Engineering 
Command.  VDMS  was  used  as  the  mobility  thread  in  the  distributed  test  environment 
demonstrations  (DTE4  and  DTE5). 

VDMS  is  part  of  the  high-fidelity  ground  platform  and  terrain  mechanics  modeling  (HGTM) 
science  and  technology  objective  (STO)  to  develop  high  fidelity,  real-time,  ground  platform 
mobility  and  terrain  models.  As  a  part  of  this  effort,  TARDEC  and  the  Army  Corps  of  Engineers 
are  developing  tire-soil  interaction  models  which  will  better  represent  tire  slippage  and  sinkage 
(Richmond,  Jones,  Creighton,  &  Ahlvin,  2004). 

Another  tool  to  consider  is  Virtual.Lab  by  LMS7  Engineering  Solutions.  Virtual.Lab  Track 
Motion  delivers  powerful  simulation  capabilities  specifically  developed  for  track  vehicle 
engineering.  With  Virtual.Lab  Track  Motion,  engineers  can  assess  the  interaction  of  the  vehicle 
with  different  terrain  profiles  to  (a)  study  stability  on  a  slope,  in  acceleration,  in  braking,  or  in  lane 
changing  (b)  evaluate  the  vehicle’s  handling,  and  (c)  optimize  driver  and  passenger  comfort.  The 
solution  computes  the  loads  between  track  links  and  suspension  parts  and  on  vehicle  bodies.  It 
also  gives  guidance  to  the  spring  and  shock  absorber  properties  and  to  the  optimal  location  of  road 
wheels,  idlers,  sprockets,  etc. 

OneSAF  test  bed  baseline  (OTB  2.0)  is  an  entity-level  battlefield  simulation  tool  used  by  many 
groups  in  the  U.S.  Army.  The  behavior  of  each  entity  is  controlled  by  a  collection  of  function 
libraries  that  handle  low-level  functions  such  as  movement,  sensory  processing,  or  weapon 
control.  Ground  vehicles  are  represented  as  simple  bodies  having  length,  width,  and  mass  with 
tracked  or  wheeled  dynamics.  The  baseline  OTB  tool  does  not  represent  vehicle  rollover; 
instead,  the  dynamics  library  limits  maximum  possible  steering  angle. 

7LMS  is  not  an  acronym. 
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While  OTB  and  other  battlefield  simulation  tools  have  limited  value  in  developing  engineering 
tests  of  vehicle  rollover,  they  are  important  in  determining  the  effect  of  vehicle  rollover  on 
overall  system  of  system  performance  in  realistic  battlefield  missions.  As  a  part  of  our  work,  we 
extended  a  version  of  OTB  2.0  to  allow  vehicle  rollover  using  the  simple  static  stability 
relationship  provided  in  equation  3.  This  rollover  extension  required  the  vehicle  definition  files 
to  be  modified  to  include  the  c.g.  height.  The  side  slope  of  the  terrain  was  used  as  an  estimate  of 
vehicle  yaw.  Once  the  conditions  for  equation  3  are  satisfied,  the  vehicle  status  changes  from 
healthy  to  mobility  killed.  Figure  6  shows  two  similar  vehicles  executing  a  slowly  increasing 
steering  maneuver  on  flat  ground.  The  vehicle  on  the  right  is  more  sensitive  to  rollover  because 
it  has  a  higher  c.g.  This  maneuver  is  somewhat  contrived,  but  with  the  SSF  equation,  researchers 
could  investigate  the  effect  of  vehicle  load  on  overall  mission  performance. 
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Figure  6.  An  OTB  simulation  comparing  two  similar  vehicles  executing  slowly  increasing  steering  maneuvers 
on  flat  ground. 


The  OneSAF  Objective  System  is  expected  to  be  released  in  the  spring  of  2006.  It  is  expected  to 
use  the  Standard  Mobility  Model  Application  Programming  Interface  (STNDMob  API). 
STNDMob  API  is  being  developed  to  consistently  represent  vehicle  mobility  for  systems  in 
battlefield  simulations  based  on  the  North  Atlantic  Treaty  Organization  (NATO)  Reference 
Mobility  Model  (NRMM)  (Baylot  et  ah,  2005).  STNDMob  provides  reasonable  speed  limits  for 
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vehicles  based  on  vehicle  type  and  the  physical  interaction  between  the  vehicle  and  the  terrain 
surface.  STNDMob  uses  the  SSF  equations  to  determine  the  maximum  safe  speed. 


7.  Virtual  Environment  Requirements 


In  this  section,  we  discuss  the  requirements  that  rollover  modeling  imposes  on  the  virtual 
environment.  We  address  two  separate  issues:  simulating  rollover  propensity  test  maneuvers 
and  simulating  rollovers  related  to  realistic  battlefield  driving. 

The  virtual  environment  required  to  simulate  rollover  propensity  test  maneuvers  is  relatively 
simple.  In  its  most  recent  studies,  NHTSA  performed  rollover  propensity  test  maneuvers  on  a 
level  test  pad  with  a  known,  constant  coefficient  of  friction.  The  digital  terrain  associated  with 
each  of  the  tools  described  in  the  previous  section  can  support  these  rollover  tests.  It  is  possible 
to  introduce  variations  into  the  basic  maneuver  tests  by  adding  hills  and  other  large  terrain 
features.  It  is  also  possible  to  vary  the  surface  composition.  VDMS,  OTB,  and  OneSAF  store 
trafficability  indices  and  terrain  attributes  for  each  elevation  post  in  the  terrain  database.  CarSim 
and  TruckSim  assign  a  friction  map  to  the  terrain  surface. 

Simulating  rollovers  related  to  realistic  battlefield  driving  requires  an  analysis  of  future  robotic 
mission  profiles.  In  its  2002  study,  the  National  Academy  of  Sciences  stated  that  future  Army 
mission  profiles  show  that  70%  to  80%  of  troop  movement  will  use  primary  or  secondary  roads 
(BAST,  2002).  Robotic  systems  are  likely  to  have  similar  mission  profiles  requiring  them  to 
maneuver  on  and  near  road  surfaces. 

In  OneSAF  and  OTB  terrain  databases,  a  road  segment  is  a  piecewise  linear  feature  consisting  of 
a  collection  of  points  on  the  battlefield,  a  road  width,  and  soil  type  for  the  road  surface  (PEO 
STRI,  2004).  CarSim  and  TruckSim  databases  describe  roads  as  collection  of  geometry  files 
specifying  the  three-dimensional  (3-D)  location  of  the  road  centerline  and  an  elevation  map  of 
the  road  surface.  The  description  also  includes  a  friction  map  of  the  road  surface.  Open  flight 
databases,  used  by  the  VDMS  tool,  can  support  detailed  road  descriptions  with  3-D  road 
geometry,  and  roadside  features  such  as  shoulders,  curbs,  and  signs.  In  their  crash  maneuver 
work,  Gopal,  Baron,  and  Shah  (2004)  replicated  various  road  geometries,  road  features  such  as 
curbs,  and  various  surfaces  such  as  asphalt,  gravel,  or  loose  soil  using  PC-Crash. 

Robots  commanded  to  follow  the  road,  like  automobiles  on  a  highway,  may  occasionally  leave 
the  road  surface  because  of  obstacles  in  the  roadway,  lack  of  perception,  or  inaccuracies  in  the 
planning  process.  With  a  high  fidelity  representation  of  the  roadway  environs,  researchers  can 
investigate  interaction  with  potential  tripping  mechanisms  such  as  gravel  shoulders,  road  edges, 
or  curbs. 
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In  an  off-road  environment,  vehicles  encounter  terrain  surfaces  with  different  textures, 
roughness,  and  friction  properties.  In  addition,  mobility  hazards  such  as  holes  and  large  rocks 
are  more  prevalent  in  an  off-road  environment.  Battlefield  simulation  tools  such  as  OTB  and 
OneSAF  use  trafficability  indices  to  determine  speed,  climb,  and  turning  rate  limitations  over 
particular  sections  of  terrain  for  each  type  of  vehicle.  The  trafficability  indices  are  derived  from 
the  NATO  NRMM  with  the  use  of  factors  such  as  soil  strength,  vegetation,  and  terrain  slope. 
Holes,  rocks,  fallen  logs,  and  other  potential  tripping  mechanisms  are  not  generally  represented 
in  these  terrain  databases.  However,  it  is  possible  to  add  tripping  features  to  OTB  terrain 
databases. 


8.  Perception  Models 


Simulations  of  test  track  maneuvers  do  not  require  sophisticated  perception  models.  The 
vehicles  drive  the  prescribed  course  on  a  smooth  level  surface  that  is  assumed  to  be  free  of 
obstructions.  However,  for  a  robotic  system  operating  in  realistic  settings,  a  rich  virtual 
environment  and  a  good  dynamics  model  are  not  sufficient  to  represent  a  robot’s  behavior  in  a 
potential  rollover  situation.  It  is  also  necessary  to  model  its  perception  of  the  environment.  A 
suitable  perception  model  represents  the  hazards  that  the  robot  “sees”  with  its  perception  system, 
the  hazards  it  identifies,  and  the  amount  of  time  the  perception  process  takes. 

Robotic  systems  use  a  variety  of  systems  to  perceive  the  environment.  These  include  scanning 
laser  radar  systems,  visible  and  infrared  cameras,  acoustic  sensors,  and  radar  systems.  First 
principle  physics  models  of  these  sensors  require  the  shape,  composition  and  density  of  obstacles 
in  the  environment,  atmospheric  models,  and  models  of  the  sensor  infonnation  collection 
process.  Such  models  do  exist,  but  they  require  information  that  might  be  difficult  to  collect  for 
a  variety  of  battlefield  environments. 

Another  approach  is  to  model  the  “end  product”  of  the  perception  system.  This  would  be  the 
information  the  robot  uses  to  drive,  such  as  a  world  map  or,  in  the  case  of  a  tele-operated  system, 
the  infonnation  displayed  to  the  operator  as  s/he  remotely  drives  the  robot.  Such  models  provide 
the  probability  of  detection  for  features  in  the  environment  as  a  function  of  obstacle  range, 
orientation,  vehicle  speed,  weather  conditions,  and  other  factors.  Ideally,  these  models  would  be 
built  with  the  use  of  data  collected  by  organizations  building  robotic  driving  sensors.  However, 
generalized  functions  could  be  used  in  conjunction  with  simulation  studies  to  look  at  the 
sensitivity  of  rollover  propensity  to  driving  sensor  performance. 
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9.  Recommendations 


In  the  previous  sections  of  this  report,  we  analyzed  the  rollover  problem,  identified  some 
potential  anti-rollover  systems,  and  discussed  virtual  simulation  tools  that  could  be  used  in 
rollover  research.  In  this  section,  we  make  some  recommendations  for  a  rollover  testing 
program.  We  recommend  three  levels  of  testing.  First,  laboratory  measures  such  as  the  SSF  and 
the  CSV  provide  insight  into  the  inherent  stability  of  the  platfonn.  On  the  sub-system  level,  we 
need  tests  to  verify  that  the  anti-rollover  system  works  as  the  designer  intends.  More 
importantly,  we  need  to  establish  the  rollover  propensity  for  the  robotic  vehicle,  especially  in  the 
off-road  environment  where  we  do  not  have  historic  data. 

At  the  subsystem  level,  we  need  to  verity  that  the  anti-rollover  system  triggers  in  time  to  prevent 
an  accident.  In  the  previous  discussion,  there  were  two  levels  of  anti-rollover  prevention:  those 
connected  with  the  vehicle  that  engage  in  a  near-rollover  condition  and  those  connected  with  the 
“driver”  that  reduce  the  probability  of  encountering  a  near-rollover  condition.  Vehicle-based 
systems,  including  rollover  warning  sensors  and  differential  braking  systems,  activate  in 
response  to  unacceptable  yaw  or  roll  rates.  The  NHTSA  maneuvers  provide  high  yaw  rates  for 
stimulating  the  anti-roll  system.  Ramp  and  hill  maneuvers  that  elevate  one  side  of  the  vehicle 
could  provide  high  roll  rate  stimulation.  Note  that  these  maneuver  tests  use  steering  machines 
and  auto-pilots  to  ensure  that  vehicles  drive  the  same  path.  Maneuvers  along  pre-determined 
paths  do  not  provide  useful  information  for  “driver” -based  systems  such  as  terrain  adaptive 
planning  algorithms.  Driver-based  systems  adjust  the  vehicle  path  to  reduce  the  probability  of 
encountering  a  near-rollover  condition.  Handling  tests,  such  as  the  slalom  maneuver  shown  in 
figure  7,  in  which  the  vehicle  picks  its  own  path  are  more  suitable  for  testing  these  systems. 
Performing  the  vehicle -based  and  driving-based  maneuvers  on  a  test  track  increases  the 
repeatability  of  these  tests. 

It  is  essential  to  protect  vehicles  and  personnel  during  testing.  Vehicle-in-the-loop  systems  such 
as  ATC’s  Roadway  Simulator  offer  a  safe  alternative  to  testing  on  tracks  for  wheeled  vehicles 
that  weigh  more  than  3000  lb.  However,  the  roadway  simulator  must  be  supported  by  detailed 
vehicle  models.  Testing  autonomous  systems  on  the  roadway  simulator  may  be  possible.  Right 
now,  an  autopilot  drives  vehicle  on  the  roadway  simulator.  For  autonomous  systems,  that 
autopilot  needs  to  be  replaced  with  the  robotic  control  system.  The  most  challenging  task  will  be 
providing  artificial  sensor  information  for  the  robotic  sensing  system. 

Simulation  tools  are  useful  for  testing  vehicle -based  and  driver-based  systems.  As  we  pointed 
out  in  our  earlier  discussions,  the  automotive  industry  extensively  employs  simulation  tools  such 
as  CarSim  and  PC-Crash  to  reduce  the  time  and  cost  of  its  safety  testing  programs.  These  tools 
require  detailed  models  of  each  vehicle  and  each  anti-rollover  system  to  be  tested.  These  models 
can  be  built  by  the  tester  or  supplied  by  the  developer.  Linking  driving  algorithms  to  simulation 
tools  allows  researchers  to  examine  driver-based  anti-rollover  systems.  For  slalom  tests, 
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simulation  tools  can  help  determine  maneuver  speeds  and  the  spacing  and  width  of  the  gates.  By 
monitoring  the  driving  algorithm  during  simulated  test  maneuvers,  testers  can  examine  the 
planning  process. 


Figure  7.  A  conceptual  slalom  course  to  test  autonomous  systems. 

Focusing  on  the  subsystem  level  of  testing  may  demonstrate  that  an  anti-rollover  algorithm 
functions  properly  on  improved  road  surfaces,  but  it  does  not  fully  address  performance  in  the 
complex  off-road  environment.  Actual  system-level  performance  is  a  combination  of  vehicle 
properties,  driver  skill,  and  the  complexity  of  the  environment.  While  this  is  certainly  equally 
true  for  manned  systems  operating  on  roads,  we  cannot  make  the  assumption  that  the  surface  is 
supposed  to  be  flat  and  smooth  and  that  there  are  “representative”  maneuvers  that  can  be  tested. 
Rollover  propensity,  a  measure  of  the  probability  that  a  system  will  roll  during  expected 
operating  conditions,  provides  system-level  performance  information  that  testers  can  use  to  rank 
performance  levels  for  various  systems  and  that  evaluators  could  use  to  construct  abstract  models 
for  large-scale  force-on-force  models. 

There  are  no  formalized  testing  procedures  for  measuring  rollover  propensity  in  the  off-road 
environment.  We  propose  using  simulation  tools  to  provide  an  initial  measurement  of  off-road 
rollover  propensity.  The  first  task  is  to  collect  rollover  statistics  from  vehicles  maneuvering  in 
terrain  patches  from  typical  off-road  environments.  Figure  8  illustrates  three  paths  from  the  left 
side  to  the  right  side  of  a  representative  test  patch.  The  contours  of  the  test  patch  are  shown  in 
brown,  and  obstacles  such  as  bushes  or  rocks  are  shown  in  green.  The  path  that  each  vehicle 
drives  is  constrained  by  the  waypoints,  shown  as  large  red  circles  in  the  figure.  The  exact  path 
and  the  speed  of  travel  are  controlled  by  the  planning  algorithms  or,  in  the  case  of  a  tele-operated 
system,  by  the  operator.  The  top  robot  path  is  incomplete;  the  robot  rolls  over  halfway  along  its 
intended  path. 

We  need  a  simulation  tool  that  has  a  detailed  model  of  the  vehicle  undergoing  test,  a 
representation  of  the  driver  algorithm,  and  rich  battlefield  environment.  Among  the  14-DOF 
dynamics  models,  VDMS  is  the  best  choice  because  it  can  be  easily  linked  with  a  distributed  test 
environment.  Also,  it  will  continue  to  be  improved  as  Cold  Regions  Research  and  Engineering 
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Laboratory  (CRREL)  and  TARDEC’s  tire-soil  interaction  research  matures.  The  distributed  test 
environment  is  the  most  efficient  way  to  include  the  driver  algorithms.  The  simulation  tools 
must  be  able  to  represent  a  variety  of  off-road  environments  containing  tripping  features  such  as 
rocks,  ditches,  and  fallen  logs.  Variable  friction  coefficients  and  deformable  surfaces  will  allow 
vehicles  to  slide  into  tripping  features  and  sink  into  the  terrain.  A  key  model  in  the  simulation 
tool  is  a  model  of  the  robotic  perception  system.  Using  a  distributed  simulation  tool  allows  us  to 
incorporate  robotic  perception  models  developed  by  other  test  centers  or  the  system  designer. 


Figure  8.  Three  paths  through  a  test  field. 


Simulation  tools  allow  us  to  study  vehicle  rollover  during  combat  scenarios  in  depth.  By 
analyzing  typical  vehicles  in  battlefield  scenarios,  we  can  identify  conditions  and  behaviors  that 
lead  to  rollover.  We  can  derive  candidate  rollover  tests  that  are  analogous  to  the  NHTSA 
maneuvers,  from  instances  of  rollover  and  near  rollover  in  the  simulated  vignettes  that  are 
relevant  to  actual  vehicle  use  in  battlefield  scenarios.  Potential  tests  can  be  simulated  in  a  variety 
of  battlefield  environments  to  examine  sensitivity  and  repeatability. 

For  specific  robotic  systems,  simulation  studies  provide  performance  data  in  environments  not 
addressed  by  physical  tests.  We  can  also  identify  minimum  conditions  to  consistently  cause 
rollover.  The  tester  uses  this  information  to  set  speed  profiles,  turn,  and  other  factors  so  that 
physical  tests  provide  usable  information  with  the  fewest  possible  runs. 
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10.  Conclusions 


The  anti-rollover  case  study  is  the  first  of  a  series  of  case  studies  used  to  refine  the  RIEP  process 
for  developing  tests  and  methodologies  to  evaluate  robotic  behavior  algorithms.  This  particular 
case  study  enabled  us  to  apply  our  current  four-step  RIEP  process  to  a  familiar  problem  for 
which  much  of  the  testing  methodology  had  been  previously  developed  for  manned  systems.  In 
the  first  step,  we  conducted  a  thorough  analysis  of  vehicle  rollover  using  the  extensive  body  of 
rollover  literature  from  the  transportation  safety  community.  Vehicle  rollover  is  an  important 
topic  in  the  transportation  industry,  so  there  are  several  papers  discussing  rollover  mechanisms, 
prevention  systems,  and  rollover  testing  programs. 

The  second  step  of  the  RIEP  process  is  to  find  a  simulation  environment  that  is  “rich”  enough  to 
simulate  the  task  and  test  the  proposed  algorithm.  Again,  the  automotive  industry  is  an 
important  source  of  information  about  the  use  of  simulation  tools  to  study  the  rollover  issue.  In 
this  report,  we  discussed  some  of  the  commercial  and  Government  vehicle  dynamics  simulation 
packages.  However,  the  automotive  industry’s  primary  interest  is  in  rollovers  on  or  near 
roadways.  Modeling  rollover  in  the  off-road  environment  requires  additional  models  for 
complex  terrain  surfaces  and  the  robotic  perception  process.  VDMS,  a  Government  simulation 
tool,  may  be  more  suitable  for  the  off-road  environment.  It  uses  a  common  terrain  database 
format  so  that  a  variety  of  terrains  can  be  easily  investigated.  VDMS  also  incorporates  complex 
tire-soil  interactions  using  on-going  research  models  from  TARDEC  and  CRREL.  Unlike  the 
commercial  products,  VDMS  is  already  compatible  with  other  simulation  tools  used  in 
distributed  simulation  exercises. 

In  the  third  step  of  RIEP,  we  link  a  proposed  rollover  prevention  system  to  the  simulation 
environment.  Currently,  we  do  not  have  a  proposed  system,  but  several  researchers  have  used 
simulation  environments  in  their  development  process  by  linking  a  MATLAB  rollover 
prevention  algorithm  to  one  of  the  vehicle  dynamics  simulation  packages.  It  is  possible  to  link 
an  actual  rollover  prevention  system  to  a  simulation  tool  if  vehicle  attitude  infonnation  such  as 
roll  or  pitch  is  provided.  This  information  must  be  transmitted  from  the  simulated  environment 
to  the  rollover  prevention  system. 

The  last  step  of  RIEP  is  to  design  a  set  of  meaningful  tests  and  performance  measures  to  evaluate 
a  proposed  rollover  prevention  system.  In  general,  most  of  the  tests  designed  to  evaluate  the 
rollover  risk  of  passenger  cars  apply  to  unmanned  vehicles.  Laboratory  measures  such  as  the 
SSF  and  the  CSV  establish  vehicle  characteristics.  Test  track  maneuvers  can  verify  that  an  anti¬ 
rollover  system  functions  in  a  benign  environment.  A  useful  metric  to  consider  is  rollover 
propensity  which  measures  the  likelihood  that  a  system  will  roll  in  a  single -vehicle  accident. 
Unlike  the  transportation  industry  which  has  years  of  accident  statistics,  we  do  not  have  a  lot  of 
data  on  rollover  accidents  for  robotic  systems  operating  in  the  off-road  environment.  We 
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recommend  a  series  of  simulation  studies  to  examine  rollover  propensity  in  the  off-road 
environment. 

Even  this  case  study  has  uncovered  differences  between  manned  and  unmanned  system  testing. 
Current  approaches  to  rollover  testing  for  manned  systems  concentrate  on  the  contribution  of  the 
vehicle.  For  autonomous  unmanned  systems,  the  algorithms  “driving”  the  vehicles  must  be 
tested  as  well.  Maneuvers  that  require  the  robot  to  plan  its  path  through  a  region  test  both  the 
driver  and  the  vehicle.  Here  again,  simulated  runs  can  supplement  the  information  we  can 
collect  from  physical  test  runs. 
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